High-fidelity reliability simulation of XOR-based erasure codes

Erasure codes are the means by which storage systems are typically made reliable. Recent high profile studies of disk failure and sector failures indicate that ever more fault tolerant erasure codes are needed. Many traditional RAID approaches, parity-check array codes (e.g.,EVENODD, RDP, and X-code), and MDS codes offer two and three disk fault tolerant schemes. There are also many novel erasure code proposals that provide similar fault tolerance, such as SPC codes, Weaver codes, and low-density parity-check (LDPC) codes. Such erasure codes offer different spaceefficiency and performance tradeoffs than traditional erasure codes. Unfortunately, such erasure codes are also distinguished by their irregular fault tolerance: different sets of disk and sector failures of similar sizes may or may not lead to data loss. Some RAID schemes provide irregular fault tolerance. For example, replicated stripes (RAID 10) has pairs of disk failures that lead to data loss, and others that do not. Reasoning about irregular fault tolerance and reliability is quite challenging. Hafner and Rao have developed Markov models of the reliability of some irregular erasure codes [3]. Elerath and Pecht recently concluded that simulation of a single-disk fault tolerant storage system utilizing Weibull-distributed failure rates and latent sector failures, leads to radically different MTTDL results than Markov models [1]. We go beyond the conclusions of Elerath and Pecht: we believe that Markov models cannot effectively model irregular fault tolerance because of the complexities of correctly modeling disk rebuild, and because latent sector failures and scrubbing should be included in the model [2]. We have developed the High-Fidelity Reliability (HFR) Simulator. The HFR Simulator permits the reliability evaluation of both irregular and traditional erasure codes under a single framework. To achieve high-fidelity simulation we leverage our prior work on minimal erasures [4]. Minimal erasures concisely and precisely describe the fault tolerance of an irregular erasure code. We used the HFR Simulator to perform the most comprehensive “apples to apples” comparison of the reliability of different erasure codes of which we are aware. We evaluated over ten different erasure code constructions in the same framework. In the comparison, we evaluate maximum distance separable (MDS) codes (i.e., ReedSolomon, RAID 4, RAID 6), parity-check array codes (i.e., EVENODD, RDP, X-code, SPC, and Weaver codes), and low-density parity-check (LDPC) codes.